Skip to content

Add a harmony tool parser for gpt-oss in the server - #2049

Open
Vl4dk0 wants to merge 1 commit into
ml-explore:mainfrom
Vl4dk0:harmony-tool-parser
Open

Vl4dk0 wants to merge 1 commit into
ml-explore:mainfrom
Vl4dk0:harmony-tool-parser

Conversation

@Vl4dk0

@Vl4dk0 Vl4dk0 commented Nov 10, 2026

Copy link
Copy Markdown

Summary

mlx_lm.server had no harmony tool parser, so gpt-oss returned raw <|channel|> text as content. The checkpoint chat template then rejects that history (You have passed a message containing <|channel|> tags), and the next turn 404s.

Nothing on main parses harmony tool calls. #1867 (closed, withdrawn) only split non-streaming text into content and reasoning, and put the commentary tool payload back into content.

This adds a harmony parser and selects it for gpt-oss:

  • analysis channel → reasoning
  • commentary to=functions.<name> → a tool call (recipient-first, as the chat template writes it, and channel-first, as the model emits it, including <|constrain|>json)
  • final channel and a commentary preamble → content
  • <|call|> is an extra eos, so a tool turn stops for the client
  • assistant history that replays tool_calls plus reasoning / reasoning_content (as OpenAI-compatible agent clients send it) is rewritten to the template's thinking field. A tool-call turn cannot carry both content and thinking, so a preamble is dropped when analysis is present.

The checkpoint chat template is unchanged.

Test plan

  • python -m unittest tests.test_tool_parsing
  • Tokenizer from gpt-oss-20b-MXFP4-Q8 selects the parser, round-trips a tool turn through the streaming detokenizer, and renders reasoning + tool_calls without raising
  • Multi-turn agent loop against mlx_lm.server with gpt-oss-20b-MXFP4-Q8: a file read, then a shell call, then the final answer, all HTTP 200 with no channel-tag 404
  • Eight-call tool loop with consecutive turns, no "does not support tool calling" warning

🤖 Generated with Claude Code

gpt-oss speaks in analysis, commentary, and final channels. The server
treated that text as content, so the next turn was rejected for containing
channel tags. Select the harmony parser for these models, return analysis
as reasoning and functions.* commentary as tool calls, and accept assistant
history that replays reasoning plus tool calls.

Co-Authored-By: Grok <noreply@x.ai>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant